NASA Technical Memorandum 100437 


Configuration Management issues and 
Objectives for a Real-Time Research 
Flight Test Support Facility 

Stephen Yergensen and Donald C. Rhea 


<IASA-Tfl- 10043?) CCHFlGOBAllCfc BABAGEHEBl 
ISSOES AKD OBJECTIVES FOB A BIAI-IIBE 
fiESIABCH EIIGfcl TEST SCEPCB1 I ACIi-XTY 
(KASA) 12 F CSCi 12A 


G3/59 


H88-2C832 


Unclas 

0136047 


May 1988 


NASA 

National Aeronautics and 
Space Administration 


NASA Technical Memorandum 1 00437 


Configuration Management Issues and 
Objectives for a Real-Time Research 
Flight Test Support Facility 

Stephen Yergensen and Donald C. Rhea 

Ames Research Center, Dryden Flight Research Facility, Edwards, California 


1988 


NASA 

National Aeronautics and 
Space Administration 
Ames Research Center 
Dryden Flight Research Facility 
Edwards, California 93523-5000 


CONFIGURATION MANAGEMENT ISSUES AND 
OBJECTIVES FOR A REAL-TIME RESEARCH FLIGHT TEST 

SUPPORT FACILITY 


f 




Stephen Yergensen 
DATAMAX Computer Systems 
NASA Contract: NAS2-12591 
Lancaster, California 

Donald C. Rhea 
NASA Ames Research Center 
Dryden Flight Research Facility 
Edwards, California 

Abstract 


Nomenclature 


AT 

acceptance testing 

CCB 

configuration control board 

CCR 

configuration change request 

Cl 

configuration item 

CM 

configuration management 

CMP 

configuration management plan 

DOD 

Department of Defense 

DR 

discrepancy report 

MB 

megabyte 

O&M 

operations and maintenance 

QA 

quality assurance 

QSM 

quality source manager 

SCM 

software configuration management 

TLCMP 

task life cycle management plan 

TPOC 

technical point of contact 

V&V 

verification and validation 

WATR 

Western Aeronautical Test Range 


This paper presents some of the critical issues and 
objectives pertaining to configuration management 
(CM) for the NASA Western Aeronautical Test Range 
(WATR) of Ames Research Center. The primary mis- 
sion of the WATR is to provide a capability for the con- 
duct of aeronautical research flight test through real- 
time processing and display, tracking, and communica- 
tions systems. In providing this capability, the WATR 
must maintain and enforce a configuration manage- 
ment plan (CMP) which is independent of, but com- 
plimentary to, various research flight test project con- 
figuration management systems. A primary WATR 
objective is the continued development of generic re- 
search flight test project support capability, wherein 
the reliability of WATR support provided to all project 
users is a constant priority. Therefore, the process- 
ing of configuration change requests (CCRs) for spe- 
cific research flight test project requirements must be 
evaluated within a perspective that maintains this pri- 
mary objective. 

WATR capability planners and designers must con- 
stantly endeavor to develop leading-edge system capa- 
bilities to satisfy expanding research flight test project 
objectives in an evolutionary manner. In conjunc- 
tion with this development, the WATR must evolve 
and implement corresponding configuration manage- 
ment plans, principles, and tools so that reliability, ca- 
pability integrity, maintainability, and safety criteria 
are ingrained within released WATR capabilities. 

Research flight test project management and con- 
trol decisions as well as critical real-time research mis- 
sion decisions are often based on data as supplied 
by the WATR. Hence, the integrity and reliability of 
WATR supplied data and information is of the ut- 
most importance. 


Introduction 

The Western Aeronautical Test Range (WATR) has 
experienced an almost exponential demand for in- 
creased facility capabilities. The capability of a real- 
time support facility to acquire data, perform real-time 
computations, and display the results in a readily us- 
able and understandable form is critical to the schedule 
of a research vehicle program. 

A key WATR configuration objective is to pro- 
vide delineated, flexible, user-oriented subsystems and 
workstations. Research flight test project engineers can 
have independent interactive control of these user sta- 
tions and their functional options. 
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WATR subsystems and user-oriented workstations 
receive, or have access to, copies of first line generated 
WATR data. First line data is critical, process-driving 
data whose integrity must be safeguarded. User sta- 
tions can be used for on-line research and experimen- 
tation in either simulation or real-time environments, 
without affecting the integrity of first line data. Pro- 
viding a high degree of user flexibility and interactive 
control capabilities on these user stations compounds 
design requirements. User flexibility and interactive 
control capabilities, although technologically advanta- 
geous for research, introduce a significant degree of 
human error potential. A capability design emphasis 
must therefore be placed on user interaction error path 
response and reaction logic. 

WATR subsystems and user workstations must be 
capable of invoking automated operational configura- 
tion controls as per research flight test project or mis- 
sion requirements. That is, flexibility and interactive 
control capability access for any user-designated sub- 
system or workstation can be automatically controlled 
as per user configuration control requirements. The 
objective is to accommodate variable user research and 
configuration control requirements. 

WATR first line data generation systems and des- 
ignated critical display systems are designed to mini- 
mize human interface error potential. On-line flexibil- 
ity and interactive control capabilities are not a config- 
uration design objective on these systems. Flexibility 
is provided via designated setup and pre-flight systems, 
wherein users can pre-test and check out flight require- 
ment specifications. WATR real-time first line data 
generation systems are automated so that they will 
not set up with unqualified flight requirement speci- 
fications for real-time operations. Automated qualifi- 
cation checklists are maintained by system software for 
individual research flight test projects. 

WATR real-time first line data generation systems 
provide simulation and playback capabilities wherein 
new flight requirement specifications can become qual- 
ified for real-time operations. WATR setup and desig- 
nated pre-flight systems can, in many cases (depending 
upon the requirement specification change), be used to 
qualify new flight requirement specifications. 

New flight requirement specifications that exceed 
existing WATR support capabilities can only be sat- 
isfied with the submittal of a WATR configuration 
change request (CCR) and its subsequent evalua- 
tion and processing. The WATR configuration con- 
trol board (CCB) serves as the initial evaluation and 
capability release approval authority. Detailed eval- 
uation, requirement definition, design, production, 
and test phase processes are conducted by designated 
WATR organizations. 


The WATR CCB serves as the official agent through 
which research flight test project officials can moni- 
tor CCR evaluation and processing status. A WATR 
technical point of contact (TPOC) is designated when 
CCR resource allocation authorization is approved by 
the applicable WATR organization. The TPOC col- 
lects research flight test project and user technical in- 
puts pertinent to a specific CCR. These inputs are used 
in CCR engineering evaluations, and, as feasible, are 
incorporated into supporting designs. 

The WATR endeavors to satisfy all research flight 
test project and user CCRs. However, the methods 
and processes by which these CCRs are satisfied and 
ultimately released for operations are an integral part 
of the WATR mission itself, and therefore must be inde- 
pendently administered via WATR configuration man- 
agement policies and procedures. 

This paper will detail those policies and objectives 
which most critically affect and apply to research flight 
test projects. The WATR’s ability to define, develop, 
and administer reliable yet unobtrusive configuration 
management policies and procedures so that they are a 
valued asset, by research flight test projects and facility 
users, is a challenging and complex objective. 

Approach to Configuration 
Management 

Typically, organizations have about the same regard 
for configuration management as individuals do for tax- 
ation. They recognize that it’s necessary, to some de- 
gree. Everyone, however, seems to disagree on what 
the levels of taxation should be, both in total and per 
taxable item. There is also significant disagreement as 
to what should and shouldn’t be taxed. Everyone does 
seem to agree that it costs too much. 

Configuration management is comprised of many 
supporting objectives, but above all, configuration 
management is a discipline concerned with the achieve- 
ment of quality. 

Quality is achieved by (1) engineering in quality, (2) 
reviewing out defects, and (3) testing out errors. There 
exists a lot of hand waving about what is and isn’t 
quality. Quality requirements must be specified, just 
as functional requirements must be specified. Quality 
is an interpreted intangible only so long as it is allowed 
to be an intangible. A quality product is one that meets 
its quality specifications. 

The cost of configuration management must also be 
weighed against costs avoided. Figure 1 shows typi- 
cal relative costs of error correction at various life cy- 
cle phases. For a real-time research flight test sup- 
port facility, the relative cost of error correction dur- 
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ing an operations phase might be an order of magni- 
tude higher than depicted since all research flight test 
projects might be impacted. Also, this does not take 
into account costs directly attributable to the error, 
which could be tragically incalculable. 

Determining the appropriate degree of configura- 
tion management is critical to every organization 
and project. It is also controversial. Ironically, the 
more successful a configuration management system is, 
the more it is perceived as being too costly and con- 
straining. 

The process of determining and specifying configura- 
tion quality requirements is systematic. Figure 2 shows 
addressable quality concerns and factors. 

The WATR is responsible for the development and 
maintenance of numerous systems, subsystems, com- 
ponents, and configuration items that are used by 
and support various research flight test projects. All 
WATR research flight test project support elements 
are delineated and identified as configuration items 
(CIs). A configuration item is “an aggregation of hard- 
ware/software, or any of its discrete portions, which 
satisfies an end use function and is designed for trace- 
able configuration identity” (Department of Defense 
DIR 5010.19, DOD-STD-480). 

Desired quality attributes and supporting quality 
specifications might vary from Cl to CI. It is important 
to note that an emphasis on a specific quality factor 
might have a negative effect on another quality factor. 
For example, if flexibility is emphasized, it will have 
a negative effect on reliability. Figure 3 shows quality 
factor effect relationships. 

Reliability is highly emphasized for most WATR CIs. 
WATR CIs vary significantly in reliability risk. Reli- 
ability risk determinations are made by answering the 
following questions: 

• What are the risks if this configuration item fails 
to perform as expected? 

• What are the risks if this configuration item fails 
to perform? 

• What are the risks if this configuration item is in- 
advertently or erroneously used? 

Note that these risks may vary per supported research 
flight test project. 

Reliability risk-based configuration quality is a vi- 
tal element of cost effective configuration management. 
For example, the testing requirements for a documen- 
tation readability analysis program simply shouldn’t 
be as demanding as the testing requirements for a 
dynamic engineering units conversion compiler. Yes, 
it is an exaggerated example, but the point is hope- 
fully clarified. 


A noteworthy WATR CM objective is to minimize, 
via configuration planning, the complexity and number 
of configuration items which have a high reliability risk. 
An example follows: 

At one time WATR setup software resided on 
first line WATR real-time systems. WATR real- 
time software executed as per specifications in 
project files generated by WATR setup software. 
Flight ready project files at first line WATR real- 
time systems could be changed inadvertently 
or erroneously via setup software at any time. 
Thus, the reliability risk of WATR setup soft- 
ware was quite high. Compounding this relia- 
bility risk was the fact that many times project 
files existed in two versions, flight ready and de- 
velopment. Human error potential was high. 

The WATR now provides projects with a dedi- 
cated setup development system. Some of the ca- 
pability and configuration management improve- 
ments are worth noting: 

1. WATR setup and first line systems are con- 
figured for removable 80 megabyte (MB) project 
disks which can be designated flight qualified or 
development. These project disks are convenient 
for transport between setup and first line sys- 
tems. Project disks also provide projects with 
a convenient and effective method to delineate 
flight qualified from development files. 

2. With a dedicated setup development sys- 
tem, project development can parallel real-time 
support operations. Previously, projects were 
developed on first line real-time systems when 
they became available. 

3. The dedicated setup development system 
can be used for postflight processing operations 
if first line systems are unavailable. 

4. The reliability risk of WATR setup soft- 
ware is lower since it doesn’t reside at a real-time 
system. Erroneous use of WATR setup software 
is still possible but now multiple errors must 
be made before real-time operations could be im- 
pacted. 

5. WATR software is also contained on remov- 
able 80 MB disks. The ability to swap software 
systems for testing purposes is highly valuable. 

Surprisingly enough, there was opposition 
to delineating setup and real-time systems. 

The drawback of having to transport (200 ft) 
project disks from a setup system to a real-time 
system seemed to some to be cruel and unusual 
punishment. 
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Configuration planning is indeed a critical function 
of configuration management. 

The WATR mission objective of providing capabil- 
ities for the conduct of aeronautical research flight 
test missions mandates an emphasized quality fac- 
tor of maintainability. Configuration item definitions, 
design, and performance requirements must be in- 
fused with maintainability criteria and considerations. 
The ability of the WATR to minimize system down- 
time is directly proportionate to its ability to achieve 
mission objectives. 

Keep in mind that if a WATR critical system fails, 
then all research flight test projects will be impacted. 
Flight research missions will not be conducted until the 
failure is verifiably corrected. 

Operations and maintenance (O&M) phase CM poli- 
cies and procedures are detailed later in this pa- 
per. Maintainability, however, cannot be effectively 
achieved unless it is considered a development priority. 
The way configuration items are defined and designed 
has a direct bearing on the WATR’s ability to maintain 
them. Preventive maintenance commences with devel- 
opment planning. Figure 4 shows a quality engineering 
development approach. 

An overview of the WATR total quality plan is pre- 
sented in Fig. 5. 

The last emphasized WATR CM approach we want 
to identify is automation. The WATR has developed 
automated CM tools that support and perform CM 
functions. Identification, configuration control, devel- 
opment control, status accounting, and auditing are 
some of the CM elements which are automatically con- 
trolled, monitored, or accounted for. The CM system, 
which is comprised of compatible, consistent, and log- 
ically sequenced automated CM tools, is known as our 
quality source manager (QSM). Figure 6 shows some 
of the critical QSM attributes and concepts. 

The scope of WATR activities and configurations is 
massive and diverse. Not all activities and configu- 
rations are under QSM’s watchful eye. We are con- 
vinced, however, that automated CM tools and sys- 
tems can be cost effective and significantly can improve 
CM effectiveness. We hope to develop more automated 
CM tools. 

We have identified some of the unique and empha- 
sized WATR CM objectives and approaches. How- 
ever, these described objectives and approaches are 
only a supporting subset of what we expect configu- 
ration management to accomplish. The following is a 
synopsis of expected CM accomplishments: 


1. Complete control, coordination, and direction of 
the configuration of systems, subsystems, components, 
and configuration items. 

2. Generation and delivery of configuration item 
data packages which meet functional, performance, and 
CM requirements. 

3. An exact compliance of configuration item base- 
lines during the various phases of a task or project. 

4. A verified accounting and reporting of planned 
and final CIs, Cl data, and configurations. 

5. A systematic evaluation, coordination, approval, 
and implementation of changes. 

6. Definition and control of interface relationships. 

7. An effective liaison with research flight test 
projects, WATR facility users, contractors, and 
vendors. 

Operational and Maintenance Phase 

The most crucial elements of WATR CM as re- 
lated to research flight test projects are the poli- 
cies and procedures related to operations and mainte- 
nance (O&M) phases of released WATR configurations. 
Problem-failure correction policies and procedures cer- 
tainly seem to garner the most attention from research 
flight test projects and managers. 

We will detail problem-failure correction policies and 
procedures as our final topic in this paper. First, we 
would like to present an overview of the WATR’s con- 
figuration management O&M objectives: 

1. Preventive maintenance methods and procedures. 

2. Maintenance of master baselines. 

3. Rapid and accurate incorporation of changes. 

4. Rapid and accurate dissemination of updated 
documentation. 

5. Traceability of changes. 

6. Change notification methods. 

7. CCB activities. 

8. Maintenance of accurate configuration identifica- 
tion data. 

An overview of the functional flow of the change 
process is shown in Fig. 7. This overview includes 
problem-failure correction procedures. Further de- 
scription of these processes follows. 

WATR capabilities problems or failures are initially 
identified and reported via a discrepancy report (DR) 
form which can be filed by anyone. 
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All DRs are submitted to the WATR CCB for evalu- 
ation and processing. WATR engineering and main- 
tenance organizations will provide immediate trou- 
bleshooting support if a failure or anomaly might im- 
pact research flight test project or mission schedules. 
In this case DRs will be generated and submitted to the 
WATR CCB chairperson in parallel or immediately af- 
ter troubleshooting findings. 

Troubleshooting will be conducted by an assigned 
cognizant technical point of contact (TPOC). The 
TPOC will be supported by additional maintenance, 
operations, or engineering personnel as the TPOC 
deems appropriate. 

The discrepancy will be demonstrated to the TPOC 
by a cognizant user or operator. 

If the discrepancy is not reproducible, the TPOC 
will generate and submit a DR response report de- 
tailing reproduction attempt and evaluation methods. 
The WATR CCB will be responsible for determining 
whether or not further action is necessary. 

If the discrepancy is reproducible, task and configu- 
ration item status accounts will be opened for tracking 
and monitoring. 

The TPOC is responsible for problem isolation and 
“work around” options. Findings and options are to be 
documented in report form and submitted to the CCB 
and the appropriate WATR engineering organization. 

If “work around” options are available, the TPOC 
and a designated CM representative must generate a 
risk evaluation report and “work around” operational 
procedure documentation for submittal to the CCB. 

If a “work around” option is acceptable to the CCB, 
then the TPOC must demonstrate the operational pro- 
cedure to applicable users, research flight test project 
personnel, and engineering representatives. 

If found to be acceptable to all concerned parties, a 
concurrence waiver form is to be signed by applicable 
users and research flight test project representatives, 
acknowledging acceptance of a “work around” option 
until final resolution. 

To achieve final resolution, the following change pro- 
cedures are followed: 

1 . Reopen the appropriate tasks for life cycle reen- 
try at the appropriate life cycle phase. Tasks would 
typically be reentered at a design or production phase. 

2. Formally amend design and production base- 
lines and test plans as per task life cycle management 
plan (TLCMP) requirements, including all necessary 
reviews and submittals. 


3. Baseline system testing is a delineated task 
which must be reopened and amended in preparation 
of changed configuration items. 

4. Execute development baseline system testing, 
producing test results and reports for review, approval, 
and archiving. 

5. Turn over system baseline to operations orga- 
nization for acceptance. The turnover includes user 
documentation, test result and report documentation, 
product deliverables, installation documentation, and 
operation demonstrations. 

6. Operations organizations will execute research 
flight test project acceptance tests as appropriate. 

7. Formal acceptance is acknowledged by signature 
of operations organization, research flight test project, 
and WATR CCB representatives. 

8. System release consists of the following: 

A. Full distribution of documentation. 

B. Configuration management closeout audits and 
reports. 

C. Monitored permanent installation of product 
deliverables. 

D. Release baseline archiving. 

Concluding Remarks 

Configuration management is necessary. The degree 
to which it is effective, advantageous, and beneficial, 
however, can vary significantly, depending upon ap- 
proach and execution. Change control policies and 
procedures can become such a focused configuration 
management function that the overall purpose and 
value of configuration management is misplaced or 
never realized. 

Configuration management is an identifiable disci- 
pline whose scope ranges from mission planning and 
project concepts to operations monitoring and perfor- 
mance trend analysis. To be ultimately effective, it 
must be both fully integrated into planning and devel- 
opment processes and yet independently accountable 
to delineated objectives. 
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concern 
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Quality 

factor 


• How well does it use 
a resource? 

Efficiency 


• How secure is it? 

Integrity 

Performance - 
how well does 

• What confidence can be 
placed in what it does? 

Reliability 

it function? 

• How well will It perform 
under adverse con- 
ditions? 

Survivability 


• How easy is it to use? 

Usability 

Design- 

• How well does it conform 
to the requirements? 

Correctness 

how valid is 

• How easy is it to repair? 

Maintainability 

the design? 

How easy is it to verify 
its performance? 

Verifiability 


• How easy is it to expand 
or upgrade its capability 
or performance? 

Expandability 

Adaptation - 
how adaptable 
is it? 

• How easy is it to change? 

Flexibility 

• How easy is it to interface 
with another system? 

Interoperability 

• How easy is it to 
transport? 

Portability 


• How easy is it to con- 
vert for use in another 
application? 

Reusability 
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Fig . 2 Configuration quality is based on 

user's perception of quality • 
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Fig, 3 All quality requirements cannot be ex- 
cellent. 
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Fig , 4 * Quality engineering and verification 
and validation . 
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Fig . 5 Western Aeronautical Test Range 

quality plan overview. 



Fig. 6 Quality source manager overview , Western Aeronautical Test Range. 
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Fig. 7 Weetern Aeronautical Test Range change process functional flow. 
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